SignWave: Human Perception of Sign Language Video Quality as Constrained by Mobile Phone Technology

Many people have benefited from the flexibility that mobile phones provide. The Deaf Community currently has limited access to the mobile telephone network through text messaging systems. However, these force the user to communicate in English as opposed to the preferred language – American Sign Language (ASL). Mobile video phones have the potential to give deaf people the independence and freedom of mobile communication in their indigenous language. Mobile phone technology offers unique challenges for real time video transfer. Two limitations of this technology are small displays and low network bandwidth. Motivated by these two constraints, we conducted a small study with members of the Deaf Community to determine the video quality needed for effective communication. We found that general compression techniques do not achieve intelligible ASL under the constraints imposed by current mobile phone networks. More visual movement is required for videos of lower quality, even though objects in the video maintain visual significance regardless of video size or distortion. Screen size was found to be less important to viewer comprehension than distortion. INTRODUCTION Cell phones with LCD displays and the ability to transmit and play videos are rapidly becoming more popular and more widely available. Their presence in the marketplace could give deaf and hard of hearing people access to the portable conveniences of the wireless telephone network through the use of sign language. The ability to transmit video (as opposed to text or symbols) would give members of the Deaf Community the most efficient and personal means of remote communication. Some members of the Deaf Community currently use text mesFigure 1. A hypothetical mobile video phone that people might use to communicate with sign language. saging, but it is extremely cumbersome and impersonal because (a) English is not the native language of most deaf Americans (ASL is their preferred language), and (b) text messaging is much slower than signed conversations. Many deaf Americans use video relay services (where a remote interpreter translates video sign language to spoken English), but this requires equipment (a computer, camera, and internet connection) that is generally set up in the home or work place and does not scale well for mobile use. Video cell phones could potentially make the mobile phone network more universally accessible to over one million deaf or hard of hearing people. Unfortunately, the Deaf Community in America cannot yet take advantage of this new technology because even today’s best video encoders cannot produce the quality video needed for intelligible (ASL) in real time given the bandwidth constraints of the wireless telephone network. In order for deaf people to utilize video cell phones for mobile communication, a new and different video compression scheme is needed. Given the network constraints, it seems likely that a successful compression scheme will be specific to sign language. Thus, information about the ways in which people view sign language videos will be crucial to the creation of such compression methods. The purpose of this research is to investigate the effects of screen size and distortion (due to compression) on visual perception of sign language videos. We used an eye tracker to collect eye movement data while members of the Deaf